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ABSTRACT 

Following a workshop on "Innovations in Employment 
Testing that Improve Validity and Reduce Adverse Impact," the City of 
Louisville (Kentucky) implemented a strategy to develop a 
comprehensive testing and recruiting program for police recruits. To 
improve candidate expectations and preparation, the following 
activities were undertaken: intense recruitment of minorities; 
revision and validation of a Test Preparation Manual (TPM) that 
provided motivated candidates from a disadvantaged educational 
background a means of focused study; and test^-taking classes, study 
skills classes, support groups, and classes covering material in the 
TPM as well as mini-academies to answer questions about the TPM. 
Practice tests were also used to help candidates prepare. The city 
addressed the testing format and content by implementing the TPM 
approach, offering classes over the material in the TPM, and adding 
an oral examination to the process. The Angoff method was used to 
establish a cutoff for scores on the written test. The cutoff was set 
at lower standards of error, and an alternate method was used to 
establish wider bands of scores to provide more candidates for 
consideration. Several problems were encountered: getting support 
from commuiiity leaders, loss of large numbers of minority candidates 
on the polygraph, and sensational ization of problems by the press and 
community leaders. (Appendixes include a list of methods to increase 
validity and reduce adverse impact and a list of Louisville efforts 
to do so.) (YLB) 
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Dr. James Outtz presented a workshop entitled "Innovations 
In Employment Testing That Improve Validity and Reduce 
Adverse Impact" at the 1989 IPMAAC Conference in Orlando 
Florida. Attachment #1 outlines the key points covered by 
Outtz three point strategy to develop a comprehensive 
testing and recruiting strategy for Police Recruit. This 
paper summarizes how the City of Louisville implemented Dr. 
Outtz 's strategy. Attachment #2 summarizes how we are 
applying each of Dr. Outtz 's individual suggestions. 

The research efforts for this project started with a review 
of our current procedures, a review of relevant literature, 
and a survey of ether jurisdictions. The Chief Examiner 
then attended the 1989 workshop and began plans to implement 
workshop suggestions which were not -already in effect. 
Several people helped to implement chis program. Dr. Outtz 
visited the City of Louisville and discussed possible 
approaches with City of Louisville representatives. Richard 
Biddle of Biddle & Associates also visited the City of 
Louisville and offered suggestions. The Jefferson County 
School Board offered classes to interested applicants and 
the Division of Police helped Civil Service with the mini- 
academies . 



IMPROVING CANDIDATE EXPECTATIONS AND PREPARATION 

The first approach to reducing adverse impact involved 
improving candidate expectations and preparation. The City 
of Louisville began targeted selection of minorities by 
conducting intense recruiting at Colleges and Minority Job 
Fairs. We worked with Biddle & Associates in extensively 
revising and validating an existing Test Preparation Manual 
(TPM) for transportability to our area. All examination 
items were developed from the TPM and subsequently validated 
by subject matter experts from the Louisville Division of 
Police. The TPM simulates the types of materials covered in 
the Police Academy. The TPM provides motivated candidates 
from a disadvantaged educational background a means of 
focused study. Since the material in the TPM will not 
change for several years candidates who do not do well on 
their first attempt can keep the TPM and study it for an 
entire year to prepare for the next examination. 



while recruitment; test development and validation were 
underway, the Chief Examiner began working with the 
Jefferson County School Board and representatives from the 
Louisville Division of Police in improving candidate 
expectations and preparation. The Jefferson County School 
Board provided test taking classes, study skills classes, 
support groups, and classes covering the material contained 
in the TPM at the DuValle Education Center, The Chief 
Examiner attended question and answer sessions at DuValle 
and set up two mini-academies at the University of 
Louisville. The Louisville Division of Police provided 
Subject Matter Experts at the mini academies to ansv/er 
questions concerning material in the TPM. The Chief 
Examiner was available at the DuValle Education Center and 
at the mini-academies to answer questions concerning the 
testing process. 

Practice tests were also used to help the candidates 
prepare. Five weeks into the study period the Civil Service 
Board administered a practice test. Correct responses were 
posted immediately after the exam. Candidates were able to 
assess their own level of acquired knowledge in each of the 
areas the exam covered. This immediate feedback enabled 
candidates to pinpoint their areas of weakness and tailor 
their studying accordingly. The practice test was 
administered at the same location and time of day as the 
actual test in an effort to condition the candidates. The 
actual exam was administered two weeks after the practice 
test. The Jefferson county School Board also provided 
practice tests over each of the areas covered in the TPM. 



TESTING FORMAT/ CONTENT AND ADVERSE IMPACT 

We addressed the testing format and content by implementing 
the TPM approach, offering classes over the material in the 
TPM anc' adding an oral examination to the process. As 
stated earlier, differences in educational background can be 
overcome by increased studying of the material in the TPM. 
Motivated candidates can increase their chances of being 
selected by investing extra time in studying the TPM. The 
motivated candidates can also increase their chances of 
success by attending the classes and the mini-academies. 
The classes offered by the Jefferson County School Board and 
the mini-academies provided another media to communicate the 
TPM material to the candidates. This process provided tools 
for motivated candidates to use to increase their 
preparation for the examination and support groups to help 
them with their expectations. The process also closely 
simulates the way Police Recruits will go about doing their 
job of learning and training to be Police Officers. 



INTERPRETING TEST SCORES 



The Angoff method was used in establishing a cut-off on the 
written test. We set the cut-off at one standard error of 
measurement below the cut-off recommended by the Angoff 
method. There was no cut-off on the oral test. A final 
score was obtained by combining the standardized written and 
standardized oral scores. Instead of using the standard 
error of measurement in setting the bands an alternate 
method was used that would establish wider bands and 
therefore, provide more candidates for consideration. Bands 
were created by starting at the mean score and moving in 
both directions one standard deviation at a time making each 
band width one standard deviation. This method resulted in 
the certification of 286 candidates. Normal use of the rule 
of three would have only resulted in 90 candidates for the 
30 available positions. This new approach also included 
everyone with a score at or above the mean. 



PROBLEMS TO OVERCOME 

Getting support from the leaders in the community was 
difficult. Critics were skeptical and preferred to sit back 
and blame tests rather than help systematically address the 
problems and try to solve them. The Jefferson County School 
Board helped us by offering training to everyone, but we 
were unsuccessful in finding mentors to make themselves 
available to interested African American applicants. The 
person we contacted to help us with a videotape on test 
taking also failed to assist us. We are attempting to 
contact minority leaders again and the Chief Examiner has 
contacted a new person to help with the test taking 
videotape. 

Another problem which has caused some concern is the loss of 
large numbers of minority candidates on the polygraph. The 
major problem for African Americans on the polygraph is the 
section of questioning concerning drug use. The Louisville 
Division of Police does not accept anyone who has smoked 
marijuana in the last three years or taken narcotics in the 
last six years. Drug testing of probationary Police 
Recruits may be a less onerous alternative than the 
polygraph in insuring a drug free workforce. 

The frustration that we have found hardest to overcome is 
that the Press and Community Leaders sensationalize the 
problems, lay blame on the easiest target, and look for 
quick fixes. Nobody seems to realize that we are faced with 
complex problems that need to be addressed systematically. 
Leaders must step forward and provide assistance and 
training to African Americans in preparing them to compete 
in the workplace. Instead of looking for scapegoats, we 
need to be looking for answers! 
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ATTACHMENT #1 



Methods to Increase Validity and Reduce Adverse impact 
(Notes from all day workshop with Dr. James Outtz*) 



I Reducing Adverse impact by Improving candidate 
Expectations and Preparation* 

A) Spend your time selectively recruiting QUALIFIED 
minorities and preparing them for the test instead of 
the usual shotgun approach of trying to get as many 
minorities as possible to apply, 

B) Give pre-test study ipanuals and/or orientation sessions 
to describe the testing process^ how to prepare for the 
test and test taking strategies. This should increase 
the knowledge base of the applicants and reduce 
differences due to test wiseness and familiarity with 
the testing process. 

C) Give pre-test physical fitness training. 

D) Conduct pre-test mini-academies. 

E) Administer sample tests as a part of the selection 
process. This procedure gives applicants realistic 
expectations^ causes inferior candidates to self select 
themselves out of testing and helps the marginal 
applicants determine the areas in which they need help. 

F) Video presentation of pre-test information can 
accommodate more candidates at varying times and also 
insures that all candidates get consistent pre-test 
information. 

II Testing Format/Content and Adverse Impact 

Open book tests ^ visual interactive tests ^ writ cen 
tests based on visual information^ oral boards^ and job 
simulations tend to have less adverse impact than 
standard multiple choice tests. 

III Interpreting Test Scores 

A) Setting cutoffs is a value judgement and is usually a 
moot issue when a large number of candidates are listed 
from the top down in order of their scores. 

B) Banding has less adverse impact than strict rank 
ordering especially if affirmative action referrals are 
made within each band. 
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ATTACHMENT #2 



CITY OF LOUISVILLE EFFORTS TO ADDRESS DR. OUTTZ'S METHODS TO INCREASE 
VALIDITY AND REDUCE ADVERSE IMPACT 



I Reducing Adverse Impact by Improving Candidate Expectations and 
Preparation 

A) We are conducting intense recruiting at Colleges and Minority Job 
Fairs throughout the State. 

B) A study manual (TPM) is used for our entry level Police Recruit 
examination. All questions come directly from the manual. 
Candidates can overcome - weaknesses by studying longer and harder. 

The Jefferson County School Board provided test taking classes, 
study skills classes, support groups, and classes covering the 
material contained in the Test Preparation Manual. 

The Chief Examiner attended a couple of classes for the Jefferson 
County School Board to describe the testing process and answer 
questions . 

D) We conducted pre-test mini-academies to answer questions 
concerning material in the TPM. 

E) We administered a sample test and posted the correct responses on 
walls outside of the test room to provide immediate feedback to 
the candidates. 

The Jefferson County School Board gave practice tests over the 
TPM. 

F) We contacted someone to present a study and test taking 
presentation for video recording, but she did not provide 
the service. We plan to attempt this again with another 
person. 



II Testing Format/Content and Adverse Impact 

— The TPM Process was used. 

— Classes and mini-academies over the material in TPM 

An oral testing component was added to the selection procedure. 



Ill Interpreting Test Scores 

A) We set a cut-off using SME input and then lowered the cut-off 
to reduce adverse impact. 

B) We used banding. Very large bands were created in order to get 
further down on the list. 



